A general scalable and accurate decentralized level monitoring method for large-scale dynamic service provision in hybrid clouds

Hybrid cloud computing combines private clouds with geographically-distributed resources from public clouds, desktop grids or in-house gateways to provide the most flexibility of each kind of cloud platforms. Service provisioning for widearea applications such as cloud backup or cloud network games is sensitive to wide-area network metrics such as round trip time, bandwidth, or loss rates. In order to optimize the quality of the service provision in hybrid clouds, it is highly valuable for the hybrid clouds to collect detailed network metrics between participating nodes of the hybrid clouds. However, since nodes can be large-scale and dynamic, the network metrics may be diverse for different cloud services, it is challenging to increase the generality, scalability, accuracy, and the robustness of the measurement process. We propose a novel distributed level monitoring method HPM (Hierarchical Performance Measurement) satisfying these requirements. For each kind of network metric, HPM represents the degree of pairwise closeness with discrete level values inspired by the hierarchical clustering tree. HPM maps probed metric to discrete levels based on an existing distributed Kmeans clustering method that helps maximize the similarity of the network metric in the same level, which therefore optimizes the matching between pairwise levels and the real-world pairwise proximity. Furthermore, for scalability reasons, HPM computes the pairwise levels with decentralized coordinates. Each node independently maintains its low-dimensional coordinate based on a novel decen∗Corresponding author. Phone: +8613875828390. Email addresses: [email protected] (Yongquan Fu), [email protected] (Yijie Wang), [email protected] (Ernst Biersack) Preprint submitted to Future Generation Computer Systems September 6, 2012 hpmfgcsrev818.tex Click here to view linked References